Cauchy-Schwarz Inequality
The Cauchy-Schwarz inequality is an inequality relating the inner products of vectors in some inner product space.
Theorem
For all \(\boldsymbol{x}, \boldsymbol{y} \in X\), where \((X, \langle \cdot, \cdot \rangle)\) is an inner product space:
\[ |\langle \boldsymbol{x}, \boldsymbol{y} \rangle|^{2} \leq \langle \boldsymbol{x}, \boldsymbol{x} \rangle \langle \boldsymbol{y}, \boldsymbol{y} \rangle.\]
Written in terms of the norm induced by the inner product the Cauchy-Schwarz inequality is:
\[ |\langle \boldsymbol{x}, \boldsymbol{y} \rangle| \leq \|\boldsymbol{x}\| \|\boldsymbol{y}\|.\]
It is one of the most fundamental inequalities useful to prove many other inequalities in real analysis.
This is very easy to see in the case of the real dot product where it follows from the fact that \(-1 \leq \cos{\theta} \leq 1\) for all \(\theta \in \mathbb{R}\).
Proof
Let \(X\) be an inner product space and let \(\boldsymbol{x}, \boldsymbol{y} \in X\).
Then consider:
\[\begin{align*}
& 0 \leq \left\langle \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|} - \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|}, \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|} - \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|} \right\rangle \\
\implies & 0 \leq \left\langle \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|}, \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|} \right\rangle + \left\langle \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|}, - \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|} \right\rangle + \left\langle \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|}, \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|} \right\rangle + \left\langle - \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|}, \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|} \right\rangle \\
\implies & 0 \leq \left\langle \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|}, \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|} \right\rangle - 2\left\langle \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|}, \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|} \right\rangle + \left\langle - \frac{\boldsymbol{y}}{\| \boldsymbol{y} \|}, \frac{\boldsymbol{x}}{\| \boldsymbol{x} \|} \right\rangle \\
\implies & 0 \leq \frac{\langle \boldsymbol{x}, \boldsymbol{x} \rangle}{\|\boldsymbol{x}\|^{2}} - 2\frac{\langle \boldsymbol{x}, \boldsymbol{y} \rangle}{{\| \boldsymbol{x} \|} {\| \boldsymbol{y} \|}} \\
\implies & 0 \leq 2 - 2\frac{\langle \boldsymbol{x}, \boldsymbol{y} \rangle}{{\| \boldsymbol{x} \|} {\| \boldsymbol{y} \|}} \\
\implies & 1 \geq \frac{\langle \boldsymbol{x}, \boldsymbol{y} \rangle}{{\| \boldsymbol{x} \|} {\| \boldsymbol{y} \|}} \\
\implies & {\| \boldsymbol{x} \|} {\| \boldsymbol{y} \|} \geq \langle \boldsymbol{x}, \boldsymbol{y} \rangle. \\
\end{align*}\]
This is almost the desired inequality, however the absolute values are missing on the right hand side. Since \(x\) and \(y\) are arbitrary, we can replace \(x\) by \(-x\) to show that case with the absolute values still holds:
\[{\| \boldsymbol{-x} \|} {\| \boldsymbol{y} \|} \geq \langle \boldsymbol{-x}, \boldsymbol{y} \rangle \implies {\| \boldsymbol{x} \|} {\| \boldsymbol{y} \|} \geq -\langle \boldsymbol{x}, \boldsymbol{y} \rangle.\]
And then finally, we can square both sides since they are both positive:
\[|\langle \boldsymbol{x}, \boldsymbol{y} \rangle|^{2} \leq \left(\|\boldsymbol{x}\| \|\boldsymbol{y}\|\right)^{2} \implies |\langle \boldsymbol{x}, \boldsymbol{y} \rangle|^{2} \leq \left(\sqrt{\langle \boldsymbol{x}, \boldsymbol{x} \rangle} \sqrt{\langle \boldsymbol{y}, \boldsymbol{y} \rangle}\right)^{2} \implies |\langle \boldsymbol{x}, \boldsymbol{y} \rangle|^{2} \leq \langle \boldsymbol{x}, \boldsymbol{x} \rangle \langle \boldsymbol{y}, \boldsymbol{y} \rangle.\]
Given \(\boldsymbol{x} = (x_{1}, x_{2}, \dots, x_{n})\) and \(\boldsymbol{y} = (y_{1}, y_{2}, \dots, y_{n})\), using the real dot product the Cauchy-Schwarz inequality is:
\[\left|\sum_{i = 1}^{n} x_{i}y_{i}\right| \leq \left(\sum_{i - 1}^{n} x_{i}^{2}\right)^{\frac{1}{2}} \left(\sum_{i - 1}^{n} y_{i}^{2}\right)^{\frac{1}{2}}.\]
This is a special case of Holder's inequality.
